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Informed  Decisionmaking  with  Diverse 
Stakeholders  Is  Complicated 


*  Issues  are  complex 

*  Actions  have  far  reaching  impact  on  many 
organizations 

*  Meaningful  communications  among  stakeholders 
may  be  limited,  unorganized,  based  on  different 
assumptions,  and  “unofficial” 

*  Each  stakeholder  must  balance  his  focused  interest 
with  need  to  interact  with  others 

*  Requires  awareness  of  others’  needs  and  views 


RAND 
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Existing  Techniques  for  informed 
Decisionmaking  Are  Lacking 


*  Undisciplined 

*  Costly 

*  Logistically  burdensome  to  implement 

*  Ineffective 

-  Fractured  focus 

*  Independence  and  anonymity  not  supported 

*  Input  mechanism  not  balanced 

-  Written  versus  verbal 
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Agenda 


*  What  is  the  E-DEL+I  approach? 

*  Why  is  E-DEL+I  valuable? 

*  How  has  E-DEL+I  been  used — an  example  is  Smart 
Outsourcing? 


RAND 


C.  Wong,  CCRTS  Charts-4  May-06 


E-DEL+I  Is  an  Analytic  Technique  That 
Facilitates  Informed  Decisionmaking 

*  Applicable  to  complex  issues  that  involve  multiple 
dimensions 

-  Technical,  political,  military,  cost,  return  on  investment, 
legal,  or  other  aspects 

*  Can  blend  technical  expertise  and  understanding  of 
military  operations/doctrine/policy  to  arrive  at  a 
balanced  solution  acceptable  to  all  stakeholders 

*  Especially  effective  when  critical  data  must  be 
derived  from  information  that  resides  in  the 
collective  knowledge  base  of  many  individuals  and 
organizations 


RAND 


C.  Wong,  CCRTS  Charts-5  May-06 


E-DEL+I  Exercises  Are  Tailored  to  the 

Application 


*  Expert  panel 

-  Panel’s  collective  knowledge  base  spans  the  issues  to  be 
addressed 

-  Panel  is  balanced  in  representation 

*  Metric 

-  Devised  to  assess  dimensions  critical  to  the  issue 

*  Questionnaire 

-  Designed  to  solicit  assessments 

*  Standard  for  consensus 

-  Higher  than  simple  majority 
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A  Typical  E-DEL+I  Exercise  Has  Four  Rounds 


Round  1 

Assessments 
&  rationales 
based  on 
experts’ 
knowledge 
and 

background 

material 

f 

Background 

material 
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A  Typical  E-DEL+I  Exercise  Has  Four  Rounds 


Round  1  statistics 
and  rationales 
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A  Typical  E-DEL+I  Exercise  Has  Four  Rounds 


Round  1  statistics 
and  rationales 


Round  2  statistics  and 
minority  arguments 
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A  Typical  E-DEL+I  Exercise  Has  Four  Rounds 


Round  1  statistics  Round  2  statistics  and  Round  3  statistics  and 

and  rationales  minority  arguments  minority  defenses 


A  Typical  E-DEL+I  Exercise  Has  Four  Rounds 


Round  1  statistics  Round  2  statistics  and 

and  rationales  minority  arguments 


Round  3  statistics  and 
minority  defenses 


Round  1 

Assessments 
&  rationales 
based  on 
experts’ 
knowledge 
and 

background 

material 

-f - 


Background 

material 
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Round  2 

Assessments 

and 

arguments 
for  minority 
positions 

Round  3 

Assessments 

and 

defenses  of 

minority 

positions 

Round  4 

Discussions 

Discussions 

Final 

assessments 

and 

justifications 

i 

- ► 

- ► 

Legend 

Feedback 

□  Not  conducted  in  real  time 

■  Can  be  conducted  in  real-time 

□  Conducted  in  real  time 


Results 
'Consensus 
'Priorities 
Shortfalls, 
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Agenda 


*  What  is  the  E-DEL+I  approach? 

*  Why  is  E-DEL+I  valuable? 

*  How  has  E-DEL+I  been  used — an  example  is  Smart 
Outsourcing? 
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E-DEL+I  Consists  of  a  Framework  and  a 
Process  with  Buiit-in  Fiexibiiity 

*  Incorporates  structured  integration  of  diverse 
inputs 

*  Supports  electronic  exercises  enabiing  many 
experts  to  participate  from  diverse  physical 
locations 

*  Has  iterative  feedback  feature  to  encourage  a  team 
approach 

*  Includes  discussion  sessions  to  encourage 
collaborative  soiutions 

*  Allows  for  comprehensive  tracking  and  quantitative 
measures  of  priority/importance 
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E-DEL+I  Maximizes  Objectivity 


*  Independent  assessments 

*  Anonymity  of  expert  panel  members 

*  Discussion  sessions  are  facilitated  by  neutral  party 

*  Final  E-DEL+I  exercise  results  define  a  way  forward 

-  Feasible  alternatives  are  identified 

-  Relative  priority/importance  of  alternatives 

-  How  many  and  which  stakeholders  agree/disagree  and 
why 

-  Areas  of  concern  and  negotiation  points 
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E-DEL+I  Minimizes  Cost  and  Logisticai  Burden 


*  Uses  commonly  available  resources 

*  Exercise  material  sent  electronically  to  participants 

-  E-mail  with  capability  to  read  attachments 

-  Microsoft  Excel  to  complete  questionnaire 

-  Telephone  to  participate  in  discussion  sessions 

*  Exercise  is  iterative 

-  Can  take  2-3  hours  or  activities  can  be  spaced  over  weeks 

-  Participation  requires  filling  out  questionnaire  for  each 
round  and  engaging  in  discussion  sessions 

*  Past  exercises  used  7  to  24  teams  of  participants 
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Agenda 


*  What  is  the  E-DEL+I  approach? 

*  Why  is  E-DEL+I  valuable? 

*  How  has  E-DEL+I  been  used — an  example  is  Smart 
Outsourcing? 
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Example:  E-DEL+I  Smart  Outsourcing  Exercise 


*  Project  purpose:  How  can  the  Army  accomplish 
more  with  its  research  dollars? 

*  Approach:  Use  E-DEL+I  to  place  Army 
technologies  on  a  market  breadth-Army  utility 
framework 

*  Expert  panel:  13  members  in  13  physical  locations 

*  Implementation:  Round  1 

-  Not  in  real  time 

-  Designed  to  encourage  participants  to  review  background 
material  and  familiarize  themselves  with  Excel  format 

*  Implementation:  Rounds  2,  3,  4  with  discussions 

-  Real  time  with  e-mail  file  transmission  and  conference  call 
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Example:  E-DEL+I  Smart  Outsourcing  Metric 


High 


Army  Utility 


Low 


0] 


Lead  -  Army  performs 
research  in-house 


Join  -  Army  joint 
research  with  other  agencies. 


Initiate  -  Army  initiates 
collaboration  with  industry. 


Participate  -  Army 
participates  in  industry  efforts. 


Monitor  -  Army  monitors 
industry  efforts. 


► 


Joint 


Generic 
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Army  unique 


Market  Breadth 
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Example:  Smart  Outsourcing  Questionnaire 

Directions:  Please  place  the  basic  technologies  In  the  framework  domains  using  the  numbers  1,  2,  3,  4,  &  5  according  to  the  following 
rating  scale. 

1 1=  Lead  -  Technology  has  limited  Industry  appeal.  Army  performs  research  In-house. 

2  =  Join  -  Technology  of  Interest  to  other  military  or  government  agency.  Army  performs 
research  jointly  with  other  agencies. 

3I=  Initiate  -  Technology  of  moderate  to  high  Army  utility  appeals  to  Industry.  Army 
collaborates  with  Industry  In  R&D. 

4  =  Participate  -  Technology  of  moderate  or  low  Army  utility  appeals  to  Industry.  Army 
collaborates  with  Industry  In  R&D. 

5  =  Monitor  -  Technology  of  moderate  to  low  Army  utility  has  high  Industry  appeal.  R&D 
performed  by  Industry  with  little  or  no  Army  resources. 


The  Army  Utility  -  Market  Breadth  Framework 


Army  unique  Joint  Generic 

Market  Breadth 


Statistical  Feedback  from 

Round  1  Responses 

Mode(s) 

Mean 

Median 

3 

2.615385 

3 

3 

2.769231 

3 

2 

1.692308 

2 

2,  3 

2.615385 

3 

9 

4 

4 

3 

3 

3 

3 

3.538462 

3 

2.923077 

3 

2 

2.692308 

3 

4 

2.923077 

3 

4 

3 

3 

4 

2.923077 

3 

FY2001  Army  Technology 


Propulsion  &  Vehicular  Mobility 


Materials  &  Mechanics 


Ballistics 


Air  Mobility 


[Applied  Physics 


Information  &  Communications 


Medical 


Engineering  &  Construction 


Sensors 


Environment  &  Soil 


Human  Engineering 


Personnel  Performance,  Training,  &  Equipment 
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Example:  Smart  Outsourcing  E-DEL+I 

Exercise  Results 


High 


Army  Utility 


Low 


Propulsion  &  Vehicular  Mobility 
Materials  &  Mechanics 
Information  &  Communications 
Engineering  &  Construction 
Sensors 


Army  Unique 


Joint 

Market  Breadth 


Generic 
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Selected  E-DEL+I  Applications 


*  Development  of  smart  outsourcing  strategies  for 
the  Army 

*  identification  of  affordable  technologies  for  the 
Army 

*  Evaiuation  of  alternative  organizational  structures 
for  Army  Laboratories 

*  Assessment  of  alternative  strategic  directions  for 
the  Army 

*  Specification  of  investment  portfoiio  management 
for  the  Navy 

*  Functionai-Area  Analysis  for  Net-Centric 
Operational  Environment 
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Summary 


*  Informed  decisionmaking  is  difficuit 

*  E-DEL+I  technique  faciiitates  informed 
decisionmaking 

-  Minimizes  cost  and  logistical  burden 

-  Maximizes  objectivity 

-  Incorporates  built-in  flexibility 

-  Tailored  to  application 

*  E-DEL+I  has  been  successfully  used  to  define  ways 
forward  in  a  variety  of  projects 
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Carolyn_Wong@RAND.org 


